Clustering or data grouping is one of the fundamental data processing activities. Clustering seeks to uncover otherwise hidden relationships between data objects with the goal of using the relationships to predict outcomes based on new data objects. For example, by identifying clusters in a set of patient data, an analyst can identify subgroups of patients with different success rates to specific treatments based on patients' data. The treatment plan for a new patient can then be based on the relationship between the new patient's data and the data for patients in the various subgroups, thus maximizing the success probability for the selected treatment regimen.
Clustering, as a data analysis tool, creates groups of data that are “close” together, where “close” implies a distance metric. Distance calculations used in clustering are defined by an analyst for the type of data based on the analyst's subjective intuition and/or experience about the similarity of the data. In some clustering techniques, the analyst selects a number of clusters to be created. Thus, the analyst's bias is present in some form in the resulting clustering, which may be overfit to existing data and produce arbitrarily uncertain results on new data.